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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

• If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.O. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )M Responsive to comnnunication(s) filed on 26 January 2004 . 
2a)S This action is FINAL. 2b)n This action is non-final. 

3) D Since this application is in condition for allowance except for fomnal matters, prosecution as to the merits is 

closed in accordance with the practice under £x parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) M Claim(s) 7-45 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) [3 Claim(s) 1-45 is/are rejected. 

Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) n The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 26 January 2004 is/are: a)|3 accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action orfomi PTO-152. 

Priority under 35 U.S.C. § 119 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (0- 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Response to Arguments 

1. Applicant's arguments on page 11, first paragraph, filed January 26, 2004, with respect to 
Figure 7 have been fully considered and are persuasive. The objection of Figure 7 has been 
withdrawn. 

2. Applicant's arguments on page 11, third paragraph, filed January 26, 2004, with respect 
to claim 34 have been fully considered and are persuasive. The 35 U.S.C. 1 12, 2"*^ paragraph 
rejection of claim 34 has been withdrawn. 

3. Applicant's arguments on pages 11-13, filed January 26, 2004, with respect to Kinjo in 
view of Cosatto have been fiilly considered but they are not persuasive. Page 11, last paragraph 
states, "the examiner states that neither Kinjo nor Cosatto disclose performing a logical AND on 
said first area and said second area, as recited in claims 1, 23, 34, and 45. Instead the Examiner 
is taking Official Notice of this missing element." The examiner respectfully disagrees. The 
examiner has not taken Official Notice of the logical AND operation. The only place the words 
"Official Notice" appear in the previous Office Action is at paragraph 14 regarding claims 3,5, 
14, 16, 25, 27, 36, and 38 with respect to the fact that determining thresholds by normalization is 
well known in the art (also listed below). The examiner realizes that neither Kinjo nor Cosatto 
explicitly recite the words "logical AND," but Cosatto clearly shows the use of shape and color 
channels to produce a list of areas which may contain head outlines and facial features wherein 
objects are located using information obtained from several channels to evaluate the quality of 
individual features as well as combinations of features including shape, color, and motion 
(column 5, line 47-54). Also, Cosatto discloses that the use of multiple algorithms or classifiers, 
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wherein several different classifiers evaluate an object independently and are then combined in a 
final step, increases the accuracy of the tracked output, extracts enhanced detail from the 
collected images, and increases the robustness of the tracked output, which is an improvement 
over that of simple color segmentation (column 2, line 8-28). The examiner believes that the 
essence of applicant's invention Ues in determining areas of low color gradient, determining 
areas of high intensity, and determining areas that have both, as recited in claim 12 where there 
is no mention of a logical AND operation. In other words, finding the combination of areas 
having both low color gradient and high intensity, and finally selecting portions of the combined 
areas with suitable hue saturation to form a candidate patch for a face. The examiner beUeves 
that Cosatto discloses the claimed combination of features (which is effectively equivalent to the 
logical AND operation). The examiner has cited additional references that disclose the claimed 
logical AND operation (see attached form PTO-892 and included references). 

Drawings 

4. The drawings were received on January 26, 2004. These drawings are acceptable. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 1-5, 8, 12-16, 19, 23-27, 30, 34-38, 41, and 45 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Kinjo (USPN 5,629,752, previously cited) in view of Cosatto 
et al. (USPN 5,864,630, previously cited, "Cosatto"). 
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7. Regarding claims 1, 23, and 45, Kinjo discloses an apparatus and method of determining 
at least one candidate patch for human faces in a color graphic image (column 19, line 18-25), 
comprising: determining a first area of the color graphic image wherein a color gradient 
(changing differential values) has a low value (exclude the regions in which differential values 
change regularly; column 18, line 65 to column 19, line 17); determining a second area of the 
color graphic image (white region) wherein an intensity value has a high value (corresponding to 
a low pixel density value; column 17, line 33-42). 

8. Kinjo does not disclose performing a logical AND on said first area and said and second 
areas to determine a third area of the color graphic image, or selecting portions of the third area 
with suitable hue saturation to form said at least one candidate patch. Cosatto discloses a multi- 
modal method for locating objects in images using information obtained from several channels to 
evaluate the quality of individual features, as well as combinations of features including shape, 
color, and motion (column 5, line 47-54). It is also disclosed that the data produced by each 
channel comprises a list of areas which may contain head outlines and facial features (column 5, 
line 66-column 6, line 2), which are compared, evaluated, and integrated by a classifier which 
assigns a measure of confidence for each feature and each combination of features to determine 
which channel or combination should be used to arrive at the final result (column 6, line 20-34). 
While Cosatto does not exphcitly disclose that a logical AND is performed on a first area and a 
second area to determine a third area of the color graphic image, it is disclosed that areas from 
each feature channel are compared, evaluated, and integrated such that areas with the desired 
features are found having the combined features (column 6, Hne 58-66). Cosatto further 
discloses that an area with suitable hue saturation (skin color) is selected to form at least one 
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candidate patch (column 13, line 14-29). It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to locate objects in images using areas found using 
combined results ("logical AND") from several feature channels and selecting areas with suitable 
hue saturation to form at least one candidate patch as taught by Cosatto in order to enable a 
tracking system to extract enhanced detail from the collected images thereby improving the 
accuracy and robustness of the tracked output over that of simple color segmentation as well as 
providing information to the tracking system which may be used to train the system for 
subsequent tracking (column 2, line 8-28). 

9. Regarding claim 12, Kinjo discloses a system configured to determine at least one 
location of a human face in a color graphic image (column 19, line 18-25), comprising: a color 
gradient map of the color graphic image configured to indicate true where a color gradient 
(changing differential values) has a low value (exclude the regions in which differential values 
change regularly; column 18, line 65 to column 19, line 17); an intensity map of the color 
graphic image configured to indicate true (white region) where an intensity value has a high 
value (corresponding to a low pixel density value; column 17, line 33-42). Cosatto (see above 
discussion of claims 1, 23, and 45) discloses a combined map configured to indicate true where 
said color gradient map is true and said intensity map is true (information obtained from several 
channels are combined to find features including shape, color, and motion; column 6, line 20- 
34), and at least one candidate patch is selected from said combined map, wherein said candidate 
patches have suitable hue saturation (skin color; column 13, line 14-29). 

10. Regarding claim 34, Kinjo discloses an apparatus comprising: a processor (facial-region 
extracting unit 40) coupled to a memory (ROM) through a bus; and a detection process 
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(program) executed by the processor from the memory (column 16, hne 37-48) to cause the 
processor to determine a first area of a color graphic image wherein a color gradient (changing 
differential values) has a low value (exclude the regions in which differential values change 
regularly- column 18, line 65 to column 19, line 17); determine a second area of the color 
graphic image (white region) wherein an intensity value has a high value (corresponding to a low 
pixel density value; column 17, line 33-42). Cosatto (see above discussion of claims 1, 23, and 
45) discloses performing a logical AND (combination) on said first area and said second area 
(information obtained from several channels) to determine a third area of the color graphic image 
(feature areas including shape, color, and motion; column 6, line 20-34), and selecting portions 
of said third area with suitable hue saturation (skin color) to form at least one candidate patch 
(column 13, line 14-29). 

11. Regarding claims 2, 13, 24, and 35, Kinjo discloses that determining said first area uses a 
first threshold value comparison (column 39, line 36-44). 

12. Regarding claims 4, 15, 26, and 37, Kinjo discloses that determining said second area 
(density) uses a second threshold value comparison (column 17, Hne 33-42). 

13. Regarding claims 3, 5, 14, 16, 25, 27, 36, and 38, Kinjo does not explicitly disclose that 
said first or said second threshold is determined by normalization. The examiner takes Official 
Notice that determining thresholds by normalization is well known in the art. It would have been 
obvious to one of ordinary skill in the art at the time the invention was made to determine the 
predetermined threshold for binarization and differential density values normalization using the 
average density calculated for the overall image as taught by Kinjo (column 17, Hne 4-9). 
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14. Regarding claims 8, 19, 30, and 41, Kinjo discloses fitting an ellipse (elliptical region) to 
one of said at least one candidate patch (column 21, line 21-28). 

15. Claims 6, 7, 9-1 1, 17, 18, 20-22, 28, 29, 31-33, 39, 40, and 42-44 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Kinjo in view of Cosatto as applied to claims 1, 8, 12, 
19, 23, 30, 34, and 41 above, and further in view of Lobo et al. (USPN 5,781,650, previously 
cited, "Lobo"). 

16. Regarding claims 6, 17, 28, and 39, neither Kinjo nor Cosatto disclose that said third area 
is eroded or that said combined map includes an eroded boundary. Lobo discloses a system for 
automatic feature detection and age classification of human faces in digital images including a 
face template that is used for oval- fitting by the use of a potential image of an edge where a 
morphological operator first broadens the image's similar intensity regions, then narrows 
(erodes) the similar intensity regions in a copy of the image, and finally the narrowed image is 
subtracted fi'om the broadened image (column 4, line 44-57). It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to erode similar intensity regions as 
taught by Lobo in order to remove stray pixels and to isolate edges for detection. 

17. Regarding claims 7, 18, 29, and 40, Lobo discloses that said eroding (narrowing) is 
morphological (column 4, line 49-63). 

18. Regarding claims 9, 20, 3 1, and 42, neither Kinjo nor Cosatto explicitly disclose 
determining if said ellipse is a bad fit to said at least one candidate patch or that said ellipse 
includes a degree of fit measure. Lobo discloses an oval fitting operation including iteratively 
updating the oval center position and the oval axes half-lengths until the total energy stabilizes 
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around a minimum value and a final fit has been reached (column 5, line 39-48). It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to determine 
the degree of fit of an ellipse about a candidate patch as taught by Lobo in order to ensure that 
the ellipse/oval is properly positioned and scaled to include the optimum amount of information 
for personnel identification or the like. 

19. Regarding claims 10, 21, 32, and 43, Lobo discloses further processing said at least one 
candidate patch when said ellipse is a bad fit (iteratively update until total energy stabilizes; 
column 5, line 39-48). 

20. Regarding claims 1 1, 22, 33, and 44, Kinjo discloses determining if said one of said at 
least one candidate patch is too smooth (column 18, line 54-63). 

Conclusion 

2 1 . The prior art made of record and not reUed upon is considered pertinent to applicant's 
disclosure. 

• USPN 5,631,697 to Nishimura et al. discloses a video camera capable of automatic 
target tracking including a logical AND operation (column 7, line 13-35). 

• USPN 5,859,921 to Suzuki discloses an apparatus for processing an image of a face 
including a logical AND operation (column 3, line 49-column 4, line 42; column 37, 
line 5-67; Figures 39-41). 

• USPN 6,249,3 17 to Hashimoto et al. discloses an automatic exposure control 
apparatus including a logical AND operation (column 24, line 63-column 25, line 12), 
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• USPAP 2001/0044818 to Liang discloses a system and method for identifying and 
blocking pomographic and other web content on the internet including a logical AND 
operation (page 3, paragraph 59-61; Figure 7). 

• USPN 6,542,625 to Lee et al. discloses a method of detecting a specific object in an 
image signal including a logical AND operation (column 5, line 4-21), 

22, TfflS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 . 136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Ryan J Hesseltine whose telephone number is 703-306-4069. 
The examiner can normally be reached on Monday - Friday, 8:30 AM - 5 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on 703-308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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